1. Field Of The Invention
The present invention pertains to neural networks and to electronic circuits for performing the learning function. More particularly, the present invention pertains to circuits for adjusting a synaptic weight based on the product of an input signal and an error signal.
2. The Prior Art
Many schemes have been proposed to use floating gate structures as weight storage for analog neural networks. Any such network requires a synaptic update mechanism which allows the weight to be changed depending on the combination of an input signal and an error signal. The most popular update rules currently in use implement some form of gradient descent, in which the weight is decreased when the input is of the same sign as the error, and is increased when the input has a sign opposite to that of the error. This form of learning is thus inherently a four quadrant computation. The desirable properties of such an update mechanism when implemented in an analog integrated circuit are small size, and freedom from high-voltage circuitry requirements within the cell itself. The learning rate of such an update mechanism should not vary widely between circuits on the same chip.
Floating gates have been proposed as a long term storage mechanism for neural networks. Many proposed on-chip learning devices use electron tunneling as the mechanism for adjusting the weights stored on these floating gates. Tunneling presents two very difficult problems. First, tunneling across an oxide induces trapping of electrons in the oxide. These electrons create a field opposing the tunneling field. The effect of charge trapping is that the performance of a tunneling device degrades over time and in a non-uniform manner from tunneling device to tunneling device. Second, for an inter-poly tunneling device, the voltage-current characteristics can vary widely for identical devices across a single chip.